{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ONNX and OML Services - Introduction\n",
    "\n",
    "Until recently, data scientists had only a handful of tools to work with, but today there is a robust ecosystem of frameworks and hardware runtimes. While this growing toolbox is extremely useful, each framework has the potential to become a silo, lacking interoperability. Supporting interoperability requires customization, and reimplementing models for movement between frameworks can slow development by weeks or months. The Open Neural Network Exchange (`ONNX`) format was created to ease the process of model porting between frameworks, some of which may be more desirable for specific phases of the development cycle, such as faster inferencing. The idea is that you can train a model with one tool stack and then deploy it using another for inference and prediction.\n",
    "\n",
    "In `ONNX` format, the machine learning model is represented as a computational graph structure with operators and metadata describing the model. It is portable across frameworks, and every framework supporting ONNX provides implementations of these operators. The `ONNX` libraries contain tools to read and write `ONNX` models, make predictions, and draw graphs of the data flow.  \n",
    "\n",
    "The [Oracle Machine Learning Services REST API](https://blogs.oracle.com/machinelearning/introducing-oracle-machine-learning-services) (`OML Services`) is included with Oracle Machine Learning on Oracle Autonomous Database cloud service. In addition to in-database models and cognitive text, the `OML Services` REST API supports `ONNX` format model deployment through REST endpoints for regression models and classification models (both non-image models and image models). \n",
    "\n",
    "In this tutorial, you'll learn how to:\n",
    "\n",
    "- Train an open source `xgboost` model\n",
    "- Convert the model to `ONNX` format\n",
    "- Deploy the model to `OML Services` on Autonomous Database\n",
    "\n",
    "Copyright (c) 2021 Oracle Corporation\n",
    "<br>\n",
    "[The Universal Permissive License (UPL) Version 1.0](https://oss.oracle.com/licenses/upl/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Train a Python XGBoost model\n",
    "\n",
    "We will create a machine learning model that can predict average house price based upon its characteristics. We'll use the popular Boston Housing price dataset, which contains the details of 506 houses in Boston, to build a regression model. \n",
    "\n",
    "To start, import the dataset and store it in a variable called `boston`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "boston = load_boston()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `boston` variable is a dictionary, and you can view its keys using the `keys` method and its shape by using the data.shape attribute, which will return the size of the dataset in rows and columns. Use the `feature_names` attribute to return the feature names."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The description of the dataset can be viewed by printing the contents of the `.DESCR` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _boston_dataset:\n",
      "\n",
      "Boston house prices dataset\n",
      "---------------------------\n",
      "\n",
      "**Data Set Characteristics:**  \n",
      "\n",
      "    :Number of Instances: 506 \n",
      "\n",
      "    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.\n",
      "\n",
      "    :Attribute Information (in order):\n",
      "        - CRIM     per capita crime rate by town\n",
      "        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.\n",
      "        - INDUS    proportion of non-retail business acres per town\n",
      "        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n",
      "        - NOX      nitric oxides concentration (parts per 10 million)\n",
      "        - RM       average number of rooms per dwelling\n",
      "        - AGE      proportion of owner-occupied units built prior to 1940\n",
      "        - DIS      weighted distances to five Boston employment centres\n",
      "        - RAD      index of accessibility to radial highways\n",
      "        - TAX      full-value property-tax rate per $10,000\n",
      "        - PTRATIO  pupil-teacher ratio by town\n",
      "        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n",
      "        - LSTAT    % lower status of the population\n",
      "        - MEDV     Median value of owner-occupied homes in $1000's\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "\n",
      "    :Creator: Harrison, D. and Rubinfeld, D.L.\n",
      "\n",
      "This is a copy of UCI ML housing dataset.\n",
      "https://archive.ics.uci.edu/ml/machine-learning-databases/housing/\n",
      "\n",
      "\n",
      "This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.\n",
      "\n",
      "The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic\n",
      "prices and the demand for clean air', J. Environ. Economics & Management,\n",
      "vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics\n",
      "...', Wiley, 1980.   N.B. Various transformations are used in the table on\n",
      "pages 244-261 of the latter.\n",
      "\n",
      "The Boston house-price data has been used in many machine learning papers that address regression\n",
      "problems.   \n",
      "     \n",
      ".. topic:: References\n",
      "\n",
      "   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.\n",
      "   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(boston.DESCR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, separate the data into target and predictor variables. Then split the data into train and test sets.\n",
    "\n",
    "We use the `train_test_split function` from sklearn's `model_selection` module with test size equal to 30% of the data. A `random_state` is assigned for reproducibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "x, y = boston.data, boston.target\n",
    "xtrain, xtest, ytrain, ytest=train_test_split(x, y, test_size=0.30, random_state=99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the regression model, we'll use the `XGBRegressor` class of the `xgboost` package with the hyper-parameter values passed as arguments. We'll initialize the regressor object and fit the regressor to the training set. Then we'll print all of the model parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBRegressor(alpha=10, base_score=None, booster=None, colsample_bylevel=None,\n",
      "             colsample_bynode=None, colsample_bytree=0.3, gamma=None,\n",
      "             gpu_id=None, importance_type='gain', interaction_constraints=None,\n",
      "             learning_rate=0.1, max_delta_step=None, max_depth=5,\n",
      "             min_child_weight=None, missing=nan, monotone_constraints=None,\n",
      "             n_estimators=10, n_jobs=None, num_parallel_tree=None,\n",
      "             random_state=None, reg_alpha=None, reg_lambda=None,\n",
      "             scale_pos_weight=None, subsample=None, tree_method=None,\n",
      "             validate_parameters=None, verbosity=None)\n"
     ]
    }
   ],
   "source": [
    "import xgboost as xgb\n",
    "\n",
    "model = xgb.XGBRegressor(objective ='reg:squarederror', colsample_bytree = 0.3, learning_rate = 0.1,\n",
    "                max_depth = 5, alpha = 10, n_estimators = 10)\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we'll train the model using the `fit` method and make predictions using the `predict` method on the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit(xtrain, ytrain)\n",
    "\n",
    "pred = model.predict(xtest)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute the RMSE by invoking the `mean_squared_error` function from sklearn's metrics module. The RMSE for the price prediction is approximately 10.4 per $1000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE: 10.391891\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics import mean_squared_error \n",
    "\n",
    "rmse = np.sqrt(mean_squared_error(ytest, pred))\n",
    "\n",
    "print(\"RMSE: %f\" % (rmse))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Convert the model to ONNX format \n",
    "\n",
    "To convert the xgboost model to `ONNX`, we need the model in `.onnx` format, zipped together with a `metadata.json` file.\n",
    "To start, import the required libraries and set up the directories on the file system where the `ONNX` model will be created.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import onnxmltools\n",
    "import json\n",
    "from zipfile import ZipFile\n",
    "from skl2onnx.common.data_types import FloatTensorType"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "home = os.path.expanduser('~')\n",
    "target_folder = os.path.join(home, 'onnx_test' )\n",
    "try:\n",
    "    os.makedirs(target_folder)\n",
    "except:\n",
    "    pass\n",
    "os.chdir(target_folder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now define the model inputs to the `ONNX` convertion function `convert_xgboost`. scikit-learn does not store information about the training data, so it is not always possible to retrieve the number of features or their types. For this reason, `convert_xgboost` contains an argument called `initial_types` to define the model input types.\n",
    "\n",
    "For each numpy array (called a tensor in `ONNX`) passed to the model, choose a name and declare its data type and shape. Here, `float_input` is the chosen name of the input tensor. The shape is defined as `None, xtrain.shape[1]]`, the first dimension is the number of rows, and the second is the number of features. The number of rows is undefined as the the number of requested predictions is unknown at the time the model is converted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "initial_types = [('float_input', FloatTensorType([None, xtrain.shape[1]]))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the model inputs are defined, we are ready to convert the `xgboost` model to `ONNX` format.  We use `convert_xgboost` from `OnnxMLTools`and save the model to file `xgboost.onnx`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "onnx_model = onnxmltools.convert_xgboost(model, initial_types=initial_types)\n",
    "onnxmltools.utils.save_model(onnx_model, './xgboost_boston.onnx')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensure that your `metadata.json` file contains the information as listed in the table *Contents and Description of metadata.json file* in [Deploy ONNX Format Models](https://docs.oracle.com/en/database/oracle/machine-learning/omlss/omlss/use-case-onxx.html).\n",
    "\n",
    "The `function` field in the `metadata.json` file is required for all models. In this case, the value for `function` in `metadata.json` is `regression`. Add the metadata and compress the file, creating `onnx_xgboost.model.zip`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata = {\n",
    "    \"function\": \"regression\",\n",
    "}\n",
    "\n",
    "with open('./metadata.json', mode='w') as f:\n",
    "    json.dump(metadata, f)\n",
    "\n",
    "with ZipFile('./onnx_xgboost.model.zip', mode='w') as zf:\n",
    "    zf.write('./metadata.json')\n",
    "    zf.write('./xgboost_boston.onnx')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Examine the string representation of the `ONNX` model. It contains the version of `OnnxMLTools`used to create the `ONNX` model, and a text representation of the graph structure, including the input types defined earlier. Note, the model can be viewed in graphical format using [`netron`](https://github.com/lutzroeder/netron)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ir_version: 7\n",
      "producer_name: \"OnnxMLTools\"\n",
      "producer_version: \"1.7.0\"\n",
      "domain: \"onnxconverter-common\"\n",
      "model_version: 0\n",
      "doc_string: \"\"\n",
      "graph {\n",
      "  node {\n",
      "    input: \"float_input\"\n",
      "    output: \"variable\"\n",
      "    name: \"TreeEnsembleRegressor\"\n",
      "    op_type: \"TreeEnsembleRegressor\"\n",
      "    attribute {\n",
      "      name: \"base_values\"\n",
      "      floats: 0.5\n",
      "      type: FLOATS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"n_targets\"\n",
      "      i: 1\n",
      "      type: INT\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_falsenodeids\"\n",
      "      ints: 6\n",
      "      ints: 5\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 8\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 7\n",
      "      ints: 6\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 10\n",
      "      ints: 5\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 7\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 10\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 8\n",
      "      ints: 7\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 8\n",
      "      ints: 5\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 7\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 6\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 10\n",
      "      ints: 9\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 6\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 8\n",
      "      ints: 0\n",
      "      ints: 10\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 16\n",
      "      ints: 5\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 13\n",
      "      ints: 10\n",
      "      ints: 9\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 15\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 20\n",
      "      ints: 19\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 22\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 6\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 9\n",
      "      ints: 0\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_featureids\"\n",
      "      ints: 5\n",
      "      ints: 10\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 10\n",
      "      ints: 7\n",
      "      ints: 0\n",
      "      ints: 10\n",
      "      ints: 7\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 7\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 5\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 3\n",
      "      ints: 6\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 9\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 9\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_missing_value_tracks_true\"\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_modes\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"BRANCH_LT\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      strings: \"LEAF\"\n",
      "      type: STRINGS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_nodeids\"\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 13\n",
      "      ints: 14\n",
      "      ints: 15\n",
      "      ints: 16\n",
      "      ints: 17\n",
      "      ints: 18\n",
      "      ints: 19\n",
      "      ints: 20\n",
      "      ints: 21\n",
      "      ints: 22\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_treeids\"\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_truenodeids\"\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 6\n",
      "      ints: 0\n",
      "      ints: 8\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 6\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 0\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 7\n",
      "      ints: 0\n",
      "      ints: 9\n",
      "      ints: 0\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 11\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 14\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 17\n",
      "      ints: 18\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 21\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 0\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 0\n",
      "      ints: 10\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"nodes_values\"\n",
      "      floats: 7.010499954223633\n",
      "      floats: 19.900001525878906\n",
      "      floats: 6.142000198364258\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 6.6863203048706055\n",
      "      floats: 0.512499988079071\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 19.400001525878906\n",
      "      floats: 1.5606000423431396\n",
      "      floats: 0.0\n",
      "      floats: 15.949999809265137\n",
      "      floats: 1.7821500301361084\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 6.6863203048706055\n",
      "      floats: 1.484299898147583\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 6.6863203048706055\n",
      "      floats: 267.5\n",
      "      floats: 10.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 688.5\n",
      "      floats: 0.0\n",
      "      floats: 0.1671150028705597\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 6.6863203048706055\n",
      "      floats: 353.46502685546875\n",
      "      floats: 0.0\n",
      "      floats: 19.0\n",
      "      floats: 388.57000732421875\n",
      "      floats: 0.2945449948310852\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 6.940999984741211\n",
      "      floats: 0.6634999513626099\n",
      "      floats: 6.14300012588501\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.7064999938011169\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.659000039100647\n",
      "      floats: 7.393500328063965\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 6.659999847412109\n",
      "      floats: 3.9850001335144043\n",
      "      floats: 0.0\n",
      "      floats: 6.144999980926514\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.5\n",
      "      floats: 77.1500015258789\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 4.650000095367432\n",
      "      floats: 0.0\n",
      "      floats: 14.795000076293945\n",
      "      floats: 6.611999988555908\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 23.829999923706055\n",
      "      floats: 0.0\n",
      "      floats: 23.974998474121094\n",
      "      floats: 0.0\n",
      "      floats: 6.620500087738037\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.6634999513626099\n",
      "      floats: 267.5\n",
      "      floats: 373.69000244140625\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.6274999976158142\n",
      "      floats: 407.0\n",
      "      floats: 0.5950000286102295\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 688.5\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 76.71499633789062\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.7064999938011169\n",
      "      floats: 43.98500061035156\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 105.23999786376953\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 4.650000095367432\n",
      "      floats: 0.0\n",
      "      floats: 14.795000076293945\n",
      "      floats: 7.809999942779541\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 33.50379943847656\n",
      "      floats: 28.684999465942383\n",
      "      floats: 0.0\n",
      "      floats: 1.2880399227142334\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      floats: 0.0\n",
      "      type: FLOATS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"post_transform\"\n",
      "      s: \"NONE\"\n",
      "      type: STRING\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"target_ids\"\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"target_nodeids\"\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 2\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 6\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 2\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 2\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 10\n",
      "      ints: 1\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 7\n",
      "      ints: 9\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      ints: 14\n",
      "      ints: 15\n",
      "      ints: 18\n",
      "      ints: 19\n",
      "      ints: 21\n",
      "      ints: 22\n",
      "      ints: 1\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 8\n",
      "      ints: 10\n",
      "      ints: 11\n",
      "      ints: 12\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"target_treeids\"\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 0\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 1\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 2\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 3\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 4\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 5\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 6\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 7\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 8\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      ints: 9\n",
      "      type: INTS\n",
      "    }\n",
      "    attribute {\n",
      "      name: \"target_weights\"\n",
      "      floats: 1.915405511856079\n",
      "      floats: 2.4355556964874268\n",
      "      floats: 1.5856462717056274\n",
      "      floats: 3.715744733810425\n",
      "      floats: 2.371917486190796\n",
      "      floats: 1.9108270406723022\n",
      "      floats: 1.059655785560608\n",
      "      floats: 0.8039546012878418\n",
      "      floats: 1.2658114433288574\n",
      "      floats: 2.792772054672241\n",
      "      floats: 1.9590023756027222\n",
      "      floats: 3.1470119953155518\n",
      "      floats: 1.4634565114974976\n",
      "      floats: 0.955295741558075\n",
      "      floats: 1.6630821228027344\n",
      "      floats: 2.6533710956573486\n",
      "      floats: 1.6440584659576416\n",
      "      floats: 0.563005268573761\n",
      "      floats: 0.0\n",
      "      floats: 0.8612136244773865\n",
      "      floats: 0.8925301432609558\n",
      "      floats: 1.245225191116333\n",
      "      floats: 2.009716510772705\n",
      "      floats: 1.3767951726913452\n",
      "      floats: 1.8786869049072266\n",
      "      floats: 0.7763971090316772\n",
      "      floats: 1.0214990377426147\n",
      "      floats: 1.3736451864242554\n",
      "      floats: 0.45191264152526855\n",
      "      floats: 0.8366303443908691\n",
      "      floats: 2.015331983566284\n",
      "      floats: 3.107783794403076\n",
      "      floats: 0.447627454996109\n",
      "      floats: 1.7570806741714478\n",
      "      floats: 1.094647765159607\n",
      "      floats: 1.8405786752700806\n",
      "      floats: 1.1085149049758911\n",
      "      floats: 0.806056797504425\n",
      "      floats: 1.5270264148712158\n",
      "      floats: 2.426034450531006\n",
      "      floats: 0.9804679751396179\n",
      "      floats: 1.5108665227890015\n",
      "      floats: 0.6090717315673828\n",
      "      floats: 0.0\n",
      "      floats: 0.30020368099212646\n",
      "      floats: 0.0\n",
      "      floats: 0.03850425407290459\n",
      "      floats: 1.4862154722213745\n",
      "      floats: 0.968802273273468\n",
      "      floats: 1.4928501844406128\n",
      "      floats: 0.6984283924102783\n",
      "      floats: 0.1243216022849083\n",
      "      floats: 0.0\n",
      "      floats: 2.074498176574707\n",
      "      floats: 0.0\n",
      "      floats: 0.31386855244636536\n",
      "      floats: 0.18400277197360992\n",
      "      floats: 0.6825682520866394\n",
      "      floats: 2.062352418899536\n",
      "      floats: 1.08696448802948\n",
      "      floats: 0.7620741128921509\n",
      "      floats: 0.46951577067375183\n",
      "      floats: 0.0\n",
      "      floats: 0.12780119478702545\n",
      "      floats: 0.0\n",
      "      type: FLOATS\n",
      "    }\n",
      "    domain: \"ai.onnx.ml\"\n",
      "  }\n",
      "  name: \"fa66663de42f4f98bd22c947daf2e0ad\"\n",
      "  input {\n",
      "    name: \"float_input\"\n",
      "    type {\n",
      "      tensor_type {\n",
      "        elem_type: 1\n",
      "        shape {\n",
      "          dim {\n",
      "          }\n",
      "          dim {\n",
      "            dim_value: 13\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "  output {\n",
      "    name: \"variable\"\n",
      "    type {\n",
      "      tensor_type {\n",
      "        elem_type: 1\n",
      "        shape {\n",
      "          dim {\n",
      "          }\n",
      "          dim {\n",
      "            dim_value: 1\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "opset_import {\n",
      "  domain: \"ai.onnx.ml\"\n",
      "  version: 1\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(str(onnx_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now score the data using the ONNX Runtime environment to validate the ONNX model is working properly. \n",
    "\n",
    "After importing the ONNX Runtime library, load the `ONNX` model in the runtime environment, get the model metadata to map the input to the runtime model, and then retreive the first 10 predictions.\n",
    "\n",
    "Note, at the time of this writing `OML Services` supports `ONNX Runtime` version 1.4.0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction:\n",
      " [[22.379318 ]\n",
      " [21.797579 ]\n",
      " [18.714703 ]\n",
      " [16.052345 ]\n",
      " [22.825737 ]\n",
      " [15.144974 ]\n",
      " [ 7.7380514]\n",
      " [14.48102  ]\n",
      " [12.728854 ]\n",
      " [ 9.806006 ]]\n"
     ]
    }
   ],
   "source": [
    "# Import the ONNX runtime environment\n",
    "import onnxruntime as rt\n",
    "\n",
    "# Setup runtime. This instantiates an ONNX inference session and loads the persisted model.\n",
    "sess = rt.InferenceSession(\"xgboost_boston.onnx\")\n",
    "\n",
    "# Get model metadata to enable mapping of new input to the runtime model\n",
    "input_name = sess.get_inputs()[0].name\n",
    "label_name = sess.get_outputs()[0].name\n",
    "\n",
    "# Create predictions. The inputs are the xtest values, and they are being casted as type float32.\n",
    "pred_onnx = sess.run([label_name], {input_name: xtest.astype(np.float32)})[0]\n",
    "\n",
    "# Print first 10 predictions\n",
    "print(\"Prediction:\\n\", pred_onnx[0:10])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify the `ONNX` and local `scikit-learn` predictions are similar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Local predictions are: [22.37932   21.797579  18.714703  16.052345  22.825739  15.144974\n",
      "  7.7380514 14.481019  12.728853   9.806006 ]\n"
     ]
    }
   ],
   "source": [
    "test_count = 10\n",
    "\n",
    "test_cases = xtest[0:test_count]\n",
    "local_pred = model.predict(test_cases)\n",
    "\n",
    "print(f\"Local predictions are: {local_pred}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Deploy the ONNX model in OML Services\n",
    "\n",
    "`OML Services` is REST API that uses an Oracle Autonomous Database as the back end repository. The `OML Services` REST API supports the following functions for OML models and `ONNX` format models:\n",
    "\n",
    "- Storing, deleting, and listing of deployed models\n",
    "- Retrieving metadata and content of models\n",
    "- Organizing models under namespace\n",
    "- Creating, deleting, and listing of model endpoints\n",
    "- Getting model APIs\n",
    "- Scoring with models\n",
    "\n",
    "To access `OML Services` using the REST API, you must provide an access token. To authenticate and obtain an access token, we use `cURL` with the `-d` option to pass the user name and password for your `OML Services` account against the OML User Management Cloud Service REST endpoint /oauth2/v1/token.\n",
    "\n",
    "Note that while we are using `cURL`, any REST client such as Postman (or even PL/SQL) can be used with `OML Services`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exchange your OML credentials for a bearer token."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    export omlserver=ADBURL\n",
    "    export tenant=TENANCYOCID\n",
    "    export database=DBNAME\n",
    "    export username=USERNAME\n",
    "    export password=PASSWORD\n",
    "\n",
    "where `ADBURL` is the Autonomous Database URL, `TENANCYOCID` is the Autonomous Database tenancy OCID, `DBNAME` is the pluggable database name, `USERNAME` is the OML user name, and `PASSWORD` is the OML user password.\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "$ curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' \n",
    "     -d '{\"grant_type\":\"password\",    \"username\":\"'${username}'\", \"password\":\"'${password}'\"}'\n",
    "     \"${omlserver}/omlusers/tenants/${tenant}/databases/${database}/api/oauth2/v1/token\"\n",
    "\n",
    "    \n",
    "    {\"accessToken\":\"eyJhbGci....6zIw==\",\"expiresIn\":3600,\"tokenType\":\"Bearer\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copy the `accessToken` field from the response and assign it to variable token, surrounded by single quotes.\n",
    "A token has a lifecycle of 3600 seconds, or 1 hour, and it can be refreshed for up to 8 hours.\n",
    "\n",
    "    $ export token='eyJhbGci....6zIw=='"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to make the model available through the REST API, we have to save it to the OML Services repository using a `POST` request. This stores the model in the repository, generates the unique modelId, and provides the URI of the repository where the model is stored.\n",
    "\n",
    "This method takes the binary data for model, model name, description, model version, model type, model namespace, and if the model is shared among tenancy users as inputs.\n",
    "\n",
    "The Linux command line JSON parser `jq` is used to format the output. To install `jq` on your Linux REST client, run the command *sudo yum install jq*.\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "\n",
    "$ curl -X POST --header \"Authorization: Bearer $token\" \\\n",
    "       ${omlserver}/omlmod/v1/models \\\n",
    "       --header 'content-type: multipart/form-data; boundary=Boundary' \\\n",
    "       --form modelData=@onnx_xgboost.model.zip \\\n",
    "       --form modelName=onnx_model \\\n",
    "       --form modelType=ONNX \\\n",
    "       --form 'description=Saving ONNX XGBoost model' \\\n",
    "       --form 'namespace=ONNX_MODELS' \\\n",
    "       --form version=1.0 \\\n",
    "       --form shared=true  | jq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The returned value contains the model ID and reference."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "    \"modelId\": \"c8c9f7d0-3e4a-4c60-bcd7-141c9116c064\",\n",
    "      \"links\": [\n",
    "    {\n",
    "      \"rel\": \"self\",\n",
    "      \"href\": \"https://adb.us-sanjose-1.oraclecloud.com/omlmod/v1/models/c8c9f7d0-3e4a-4c60-bcd7-141c9116c064\"\n",
    "    }\n",
    "  ]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, deploy the model by creating the model scoring endpoint identified by the model ID and requested URI, `onnx_model`."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "$ curl -X POST \"${omlserver}/omlmod/v1/deployment\" \\\n",
    "       --header 'Content-Type: application/json' \\\n",
    "       --header \"Authorization: Bearer ${token}\" \\\n",
    "       --data '{\n",
    "        \"modelId\":\"c8c9f7d0-3e4a-4c60-bcd7-141c9116c064\",\n",
    "        \"uri\":\"onnx_model\"}' | jq\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This returns the model ID, URI, and a time stamp containing the deployment information:"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "{\n",
    "  \"links\": [\n",
    "    {\n",
    "      \"rel\": \"self\",\n",
    "      \"href\": \"https://adb.us-sanjose-1.oraclecloud.com/omlmod/v1/deployment/onnx_model\"\n",
    "    }\n",
    "  ],\n",
    "  \"modelId\": \"c8c9f7d0-3e4a-4c60-bcd7-141c9116c064\",\n",
    "  \"uri\": \"onnx_model\",\n",
    "  \"deployedOn\": \"2021-06-13T19:37:47.272Z\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the model is saved in the model repository, you can view it in the OML Notebooks area in Autonomous Database. Select the hamburger menu, then navigate to `Models` and look for the deployed model under `Deployments`. Select the model name to view the model metadata, and select the model URI to view the Open API specification for the model.\n",
    "\n",
    "You can also view the model endpoint details from the REST API. For example, the deployment endpoint, identified by the model URI, returns the model Id, URI, and date of deployment:"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "$ curl -X GET \"${omlserver}/omlmod/v1/deployment/onnx_model\" \\\n",
    "       --header \"Authorization: Bearer ${token}\" | jq\n",
    "       \n",
    "{\n",
    "        \"links\": [\n",
    "        {\n",
    "          \"rel\": \"self\",\n",
    "          \"href\": \"https://adb.us-sanjose-1.oraclecloud.com/omlmod/v1/deployment/onnx_model\"\n",
    "        }\n",
    "      ],\n",
    "      \"modelId\": \"c8c9f7d0-3e4a-4c60-bcd7-141c9116c064\",\n",
    "      \"uri\": \"onnx_model\",\n",
    "      \"deployedOn\": \"2021-06-13T19:37:47.272Z\"\n",
    "    }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's make predictions on the model we just deployed and compare them against the `XGBRegressor.predict`method used locally. A single score is returned:"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "$ curl -X POST \"${omlserver}/omlmod/v1/deployment/onnx_model/score\" \\\n",
    "       --header \"Authorization: Bearer ${token}\" \\\n",
    "       --header 'Content-Type: application/json' \\\n",
    "       --data '{\"inputRecords\": [{\"float_input\": [[0.00632, 18.0, 2.31, 0.0, 0.538, 6.575, 65.2, 4.0090, 1.0, 296.0, 15.3,                                                    369.90, 4.98]]}]}' | jq\n",
    "       \n",
    "{\n",
    "  \"scoringResults\": [\n",
    "    {\n",
    "      \"regression\": 20.695398330688477\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To delete the model, first delete the model endpoint, then delete the stored model using the model Id. Attempting to delete a deployed model from the `OML Services` REST API will result in an error that the model is currently deployed."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "curl -i -X DELETE \"${omlserver}/omlmod/v1/deployment/onnx_model\" \\\n",
    "--header \"Authorization: Bearer ${token}\""
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "curl -i -X DELETE \"${omlserver}/omlmod/v1/models/c8c9f7d0-3e4a-4c60-bcd7-141c9116c064\" \\\n",
    "--header \"Authorization: Bearer ${token}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To learn more about [ONNX](https://onnx.ai/) and [Oracle Machine Learning Services](https://docs.oracle.com/en/database/oracle/machine-learning/omlss/omlss/use-case-onxx.html) refer to these documentation resources and our [Ask Tom Office Hours library](https://asktom.oracle.com/pls/apex/asktom.search?office=6801)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "metadata": {
   "interpreter": {
    "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
